End-to-end multilingual ASR has become more appealing because of several reasons such as simplifying the training and deployment process and positive performance transfer from high-resource to low-resource languages. However, scaling up the number of languages, total hours, and number of unique tokens is not a trivial task. This paper explores large-scale multilingual ASR models on 70 languages. We inspect two architectures: (1) Shared embedding and output and (2) Multiple embedding and output model. In the shared model experiments, we show the importance of tokenization strategy across different languages. Later, we use our optimal tokenization strategy to train multiple embedding and output model to further improve our result. Our multilingual ASR achieves 13.9%-15.6% average WER relative improvement compared to monolingual models. We show that our multilingual ASR generalizes well on an unseen dataset and domain, achieving 9.5% and 7.5% WER on Multilingual Librispeech (MLS) with zero-shot and finetuning, respectively.
translated by 谷歌翻译
神经网络修剪可以有效地用于压缩自动语音识别（ASR）模型。但是，在多语言ASR中，执行语言不足的修剪可能会导致某些语言的严重性能降解，因为语言 - 敏捷的修剪口罩可能不符合所有语言，并丢弃了重要的语言特定参数。在这项工作中，我们提出了ASR路径，这是一种稀疏的多语言ASR模型，该模型激活了特定语言的子网络（“路径”），从而明确地学习了每种语言的参数。通过重叠的子网络，共享参数还可以通过联合多语言培训来实现较低资源语言的知识传输。我们提出了一种新型算法来学习ASR途径，并通过流式RNN-T模型评估了4种语言的建议方法。我们提出的ASR途径的表现都优于密集模型（平均-5.0％）和语言不足的修剪模型（平均-21.4％），并且与单语稀疏模型相比，低资源语言的性能更好。
translated by 谷歌翻译
Several solutions for lightweight TTS have shown promising results. Still, they either rely on a hand-crafted design that reaches non-optimum size or use a neural architecture search but often suffer training costs. We present Nix-TTS, a lightweight TTS achieved via knowledge distillation to a high-quality yet large-sized, non-autoregressive, and end-to-end (vocoder-free) TTS teacher model. Specifically, we offer module-wise distillation, enabling flexible and independent distillation to the encoder and decoder module. The resulting Nix-TTS inherited the advantageous properties of being non-autoregressive and end-to-end from the teacher, yet significantly smaller in size, with only 5.23M parameters or up to 89.34% reduction of the teacher model; it also achieves over 3.04x and 8.36x inference speedup on Intel-i7 CPU and Raspberry Pi 3B respectively and still retains a fair voice naturalness and intelligibility compared to the teacher model. We provide pretrained models and audio samples of Nix-TTS.
translated by 谷歌翻译
本文介绍了基于Wav2VEC 2.0的跨语言语音表示学习的大规模模型。我们在128种语言中培训最多2B个公共讲话音频的近半小时的型号的模型，比公共数据的数量级比最大的已知事先工作。我们的评估涵盖了广泛的任务，域，数据制度和语言，都是高低资源。在Covost-2语音翻译基准测试中，我们将先前的最先进的状态平均为7.4 BLEU超过21个翻译方向进入英语。对于语音识别，XLS-R在Babel，MLS，CommonVoice以及Voxpopuli上的最佳已知工作中提高，降低了相对的误差率14-34％。 XLS-R还在Voxlingua107语言识别上设置了新的技术状态。此外，我们表明，具有足够的模型规模，交叉思维预先预测可以在将英语演讲翻译成其他语言时才能优于英语撇印，这是一个有利于单晶的预借预制的设置。我们希望XLS-R可以帮助改善世界上更多语言的语音处理任务。
translated by 谷歌翻译
从未标记数据的代表学习一直是对人工智能研究的重大兴趣。虽然自我监督的言语代表学习在语音研究界受欢迎，但很少有效地对非语音音频任务进行了全面分析了音频表示学习。在本文中，我们提出了一种自我监督的音频表示学习方法，并将其应用于各种下游非语音音频任务。我们将众所周知的Wav2Vec 2.0框架结合起来，这在用于语音任务的自我监督学习中取得了成功，具有参数效率的构装体系结构。我们的自我监督的预培训可以减少三分之二的标记数据的需求。在Audioset基准测试中，我们达到平均平均精度（地图）得分为0.415，这是通过仅限音频自我监督的学习在此数据集上的新型最先进的。我们的微调符合子也超越了在几个下游任务上以监督方式预先培训的先前系统的性能。我们进一步讨论了预先培训和微调的重要设计考虑因素。
translated by 谷歌翻译